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ABSTRACT 



In response to changing management and budgetary strategies 
that have been driving decision making to the lowest administrative levels, 
institutional researchers have been attempting to identify indicators which 
can be used to evaluate institutional performance and progress. This paper 
reviews efforts to introduce departmental operational performance indicators 
at the University of Minnesota. A decision support schematic illustrates the 
instructional resource management tool that was developed. The diagram shows 
the relationships of the broad indicators, such as demand, fiscality, and 
productivity, to each other and to specific lower-level measures such as 
tuition revenue management, instructional cost management, curriculum 
management, enrollment and tuition management, and faculty course management. 
Two other programs, a "U2000 critical measures" process and incentives for 
managed growth, are also discussed. The former, an attempt to demonstrate to 
faculty how daily planning and operational decisions affect overall 
institutional goals, was judged to have failed. The initial attempts to 
establish an "incentives for managed growth" program (also termed 
responsibility-centered management) , a financial strategy that emphasizes 
entrepreneurial activities and encourages decision making at the lowest 
administrative levels, were disappointing, and few of the university's 
departments were deemed able to use this strategy. (CH) 
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Collegiate and Departmental Performance Indicators 

Abstract 



Largely in response to external pressures from legislatures and other funding sources, 
institutional researchers are devoting increasing efforts identifying valid indicators and 
benchmarks to evaluate institutional performance and progress. Unfortunately these measures 
are typically too nebulous, aggregated or distal for collegiate and departmental resource planning 
and allocation decisions. Without the ability to clearly translate institutional goals into 
operational decisions or policies, the probability that collegiate and departmental efforts will 
consistently contribute toward institutional goals significantly decreases. This paper identifies 
and discusses several departmental operational performance indicators for instructional resource 
management. These measures will be of greater utility for attaining institutional financial and 
programmatic objectives especially for institutions seeking new approaches to resource 
allocations such as Responsibility Centered Management. In addition, this paper suggests that 
colleges and departments will increasingly turn to interdisciplinary professionals who combine 
institutional research, financial and information technology expertise with an understanding of 
the informational needs to support daily operational decisions. 
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Overview 

The choice of this year’s (37 th Annual) Association for Institutional Research forum theme, 
“Performance Indicators: Defining Measures that Matter” reflects the increasing competition for 
public and private funding and the resulting demand for greater value and accountability from 
higher education institutions. Largely in response to these external pressures, benchmarks, 
performance indicators, and measures of varying qualitative and quantitative nature have sprung 
up on the landscape. Internal recognition of the utility of operational performance measures for 
allocating resources and managing the vast array of enterprises that typically comprise 
universities, on the other hand, has been much slower to develop. Even when the impetus for 
improved operational performance measurement occurs within institutions, initial efforts often 
focus on adapting measures developed for external constituencies to meet internal needs. 
Institutional level performance indicators have limited utility for deans and department heads - 
the administrators who make the daily operating decisions that ultimately drive the institutional 
outcomes. 

Current institutional benchmarks are important - they play a needed role in communicating 
achievements in basic terms to, and justifying or securing more revenues from, external 
constituencies. These efforts may be an effective short-term solution, however, if performance 
measurement and benchmarking activities in higher education do not extend beyond satisfying 
external constituencies, the true value-added benefits of these performance measures will be lost 
to institutions. The lack of performance indicators, designed to support internal daily operational 
decisions, must be addressed. The ability for colleges and universities to survive and flourish in 
the next millenium will require significant improvements in the strategic use of existing 
resources and large gains in the understanding and management of costs. 

Despite the critical importance of operational performance measures, the absence of sufficient 
external pressures to force the issue combined with the following internal barriers substantially 
decreases the likelihood of their development and increases the operating risks of colleges and 
universities: 



0 




5 



Collegiate and Departmental Performance Indicators 



4 



• Historical tradition of universities as loosely knit federations of independent colleges, 
departments and faculties, 

• Deeply held perceptions that good education and good management are mutually exclusive, 

• Lack of trust between faculty and central administrators, 

• Insufficient management and financial expertise, 

• Absence of easily accessible operating information. 

As colleges and departments grapple with dramatic changes in their current environment and 
search for a management paradigm sensitive to the special needs of higher education, the 
necessity for a new type of institutional research professional has become increasingly apparent. 
This need is exacerbated by the expansion of management and budgetary strategies that drive 
decision-making to the lowest level in educational institutions. Colleges and departments will 
increasingly look to interdisciplinary professionals who combine institutional research, financial, 
managerial, and information technology expertise with an understanding of the informational 
needs to support daily operational decisions. 

What drives the need for an interdisciplinary approach? There are several basic principles that 
underlie collegiate and departmental operational decision support: 

• Optimal decisions require information from multiple varied areas including curricular, 
financial, student and human resources, 

• Information must be integrated, intuitive and readily accessible to support daily decisions, 

• Communication of information must respond to the wide variance of knowledge and 
experience of collegiate and departmental staff. 

Collegiate and departmental experiences with central administrative offices, whether institutional 
research, budget, human resources or information technology, suggest these units are not well 
positioned to meet these needs. These offices tend to work at the proverbial 35,000 feet where 
collegiate and departmental trees, cars, and houses all look the same. Typical reports generated 
by central administrative units, aggregated at the organizational level and transaction-oriented, 
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often treat variations between colleges, and especially departments, as statistical noise. This 
approach can result in material differences between data provided by central administrative units 
and data needed by colleges and departments for making operating decisions. In the absence of 
skilled analytical staff, colleges and departments often lack the ability to articulate to central 
offices their decision support needs, or utilize available data to inform management decisions. 

Recognition of both central staff and this new type of professional does not reflect duplication of 
function, but rather the fact that an organization requires expertise to support managerial 
decisions throughout all levels and increasingly in colleges and departments. 

This paper will review the efforts we have made to introduce management tools for collegiate 
and departmental decision support at the University of Minnesota. Two additional relevant 
events at the University of Minnesota will also be discussed: 

• The implementation of “U2000 Critical Measures” process and, 

• The introduction of “Incentives for Managed Growth”. 

The first event refers to the efforts of central institutional researchers to establish benchmarks 
and performance indicators as part of the university strategic planning process and the failed 
attempts by a college to adapt these measures to their strategic planning and budget process. The 
second event refers to a change in the University of Minnesota budgeting process, similar to 
“Responsibility Centered Management”, or RCM (Whalen, 1991), that exacerbates the need for 
an information-rich decision support environment for college and departmental administrators. 
These two developments have been key drivers in forcing recognition of the need for both staff 
and performance indicators that address collegiate and departmental operating issues. 

In addition, operational performance indicators for instructional resource management will be 
introduced as well as the latest reports designs and user functionality available through cutting 
edge Web technology. 
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U2000 Critical Measures 



In 1993, the University of Minnesota began a process to develop benchmarks for communication 
with external and internal constituencies regarding the progress and success of the U2000 
campaign. Initial discussions between central institutional researchers and collegiate faculty and 
staff were designed to introduce the broad outlines of the process, the strategic areas of 
investment and the criteria for which the University and its units would ultimately be measured. 
The seven criteria identified were Quality, Centrality for the Academic Disciplines, Centrality 
for the State, Comparative Advantage, Efficiency, Effectiveness, and Demand. 

The College of Liberal Arts, the largest college at the University of Minnesota consisting of the 
arts, humanities and social sciences departments, responded by charging a faculty advisory group 
with identifying valid measures for which the college could be assessed along these seven 
dimensions. This process was considered to be critical because it was believed that future 
resource allocations, both to the college from the university and to the departments from the 
colleges, would reflect success and progress using these benchmarks. 

The college advisory committee quickly realized that the definitions imparted sufficient 
vagueness to create lengthy discussions of interpretations. For example, here is an abbreviated 
sample of a few themes (Infante and Kvavik, 1994): 

• “Quality refers to the level of excellence aspired to and currently provided by academic 
programs and service units. This measure is ultimately a question of how well the values for 
which the University stands are expressed in its activities — in free inquiry, effective learning, 
and useful service to the citizens of the State....”, 

• “Comparative Advantage is about the unique characteristics that make a program particularly 
appropriate to this University. It is not sufficient that programs meet an important local or 
national need, or that they be unique within the State....”, 

• “Effectiveness concerns the degree to which agreed upon and desired results are achieved”. 
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Despite the lack of consensus of the appropriateness of some of these themes, the committee 
members pressed on. The process, however, quickly hit an impasse as members of the 
committee, representing departments across the three main disciplines in the college, could not 
agree on the validity or applicability of a given measure because it inherently disadvantaged one 
discipline over another. For example, at a major research institution such as the University of 
Minnesota, the ability to gamer sponsored research funding was considered the quintessential 
measure of quality - by the social scientists. The artists and humanists were very wary of such a 
measure even when it was suggested that comparisons would be limited to similar disciplines 
within the college, the same disciplines across peer institutions, or within a discipline across 
time. The measure was retained but only for a subgroup of departments. 

Scholarly productivity was mutually recognized as a measure of quality (as well as effectiveness) 
but again, identifying a measure proved difficult. Both social scientists and humanists identified 
journal and book (or chapter) publications, but artists preferred exhibitions and performances. 
Even still, there was great debate and consternation about the “quality” of the quality measures 
(e.g. was the journal refereed, was the publication or performance internal or external, etc.). 

After a year of deliberation, over 5 1 departmental and collegiate measures, including an 
opportunity to submit a short departmental narrative of the “glories” of the unit, were identified. 
Many committee members, as their biannual terms were ending, felt the outcome and process 
were flawed and distrusted how the measures would eventually affect resource allocation 
decisions. In the end, the college settled on about a dozen measures to begin assessing in the 
first phase. Several of these measures had always been collected (e.g. number of graduate and 
undergraduate majors, faculty, student credit hours, sponsored expenditures, etc.). Other 
measures in this subgroup had not been collected, either due to the lack of available data (e.g. 
graduation placement) or the difficulty of collecting, summarizing and analyzing the data (e.g. 
faculty accomplishments from annual activity reports). As of today, no known progress has been 
made on these last two groups of indicators, and the impact of Incentives for Managed Growth 
on resource allocations has lessened the need to pursue these endeavors. 

Despite uncertainty over the synergy between Incentives for Managed Growth and the U2000 
Critical Measures process, the central process has continued with expansion of potential 
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measurement categories. Currently fourteen measurement categories have been identified during 
Phase III including the following (Whiteside, Hendel, Dunbar and Kvavik, 1997): 

• compensation position among peer institutions for faculty and administrative staff, 

• dollars in millions of total deferred renewal, 

• mean high school rank of entering freshmen, 

• percentage of students expressing satisfaction with essential student , 

• dollars of instructional direct expenditure per student relative to comparable , 

• percentage of all entering freshmen who graduate in five years, 

• listing of scholarly “highlights”. 

Three years later, colleges and departments are no closer to clearly understanding the 
relationship between their operational decisions and the overall institutional goals; yet there is no 
question that their daily planning and operational decisions ultimately affect these outcomes. 
Central strategic planning and measurement and collegiate and departmental operating decisions 
remain discrete activities with limited connection. In fact annual university requests for planning 
documents are viewed by colleges as a chore rather than an opportunity to integrate institutional 
goals and define how actions contribute to outcomes. Requests often are supplemented with 
collegiate data grids, designed to assist in the process, that either do not accurately reflect 
collegiate activities or have little overlap with actual informational needs. Higher education is 
learning to measure outcomes but learning to manage to an outcome remains beyond the grasp of 
most institutions. 

Incentives for Managed Growth 



The continuing decline in the percentage of public funding for higher education is forcing 
institutions such as the University of Minnesota to place greater reliance on tuition revenue, the 
creation of new revenue streams, cost containment, and productivity improvements to survive. 
This financial strategy emphasizes entrepreneurial activities and promotes driving incentives and 
decision making to the lowest possible levels of the institution. This shift requires changes in 
budgetary strategies that fall under the rubric of “Responsibility Centered Management”, 
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“Responsibility Centered Budgeting”, or in the case of the University of Minnesota, “Incentives 
for Managed Growth” (“Incentives for Managed Growth”, 1996). While the opportunity for a 
unit to grow itself out of financial problems rather than cut itself out is inherently more positive, 
this budgeting approach, however, incurs some high risks that are not always clearly recognized. 
The primary risk resulting from this decision is that institutions are inexorably moving towards 
increasing privatization of public universities without developing the necessary expertise, 
management depth, and information systems to support this strategic decision. 

This philosophical shift has repercussions throughout the organization. Decentralized units reap 
the direct revenue impact of their actions - both positive and negative. Unfortunately, decisions 
that have positive benefits to the unit may have deleterious effects on the rest of the institution. 
University central administrators have less of an impact to shape the overall funding structure of 
a college or department while remaining dependent upon those units to fulfill institutional 
missions and goals. The presence of collegiate and departmental operational performance 
indicators becomes more critical both to provide decision support to the units and to inform 
central units of the impact of these decentralized actions. Without the presence of critical 
information, especially early warning signals, to manage daily decisions, college and 
departmental administrators may make decisions that not only place their unit in financial 
jeopardy but the overall institution as well. 

During a working group discussion, attended by both collegiate and central administrative staff, 
of the information needs required for implementation of this budgeting approach, the question 
was asked whether existing institutional data warehouses contained sufficient information. 
Central staff felt sufficient data existed, several collegiate staff did not know, while we felt there 
was a definite information shortfall. All the responses were accurate, given individual 
perspectives and experiences, and reflect disparate views on the information needed for colleges 
and departments to survive. 

After a considerable amount of debate and discussion, the University of Minnesota announced in 
the summer of 1996, that a decision had been made to move forward with “Incentives for 
Managed Growth”. Implementation would begin starting July 1, 1997 (FY98). This accelerated 
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timeline left little or no opportunity for development of the necessary information base and 
consensus on the quality of data to be used, much less strategies for delivering the information 
and training administrators and staff. The colleges under the direction of the Provost’s Office for 
Arts, Sciences, and Engineering, however, were relatively advantaged since staff within the 
office had recognized the need to provide this information up to three years earlier. Most 
colleges have not yet begun to recognize the need much less develop systems to deliver the 
content of information and the type of tools required to meet the needs created by Incentives for 
Managed Growth and the mission of the colleges and departments. 

Instructional Resource Management 

There were several reasons that led to the decision to develop instructional resource management 
tools. The first reflects one of the primary missions of the colleges in Arts, Science, and 
Engineering (including, the College of Biological Sciences, College of Liberal Arts, General 
College, and the Institute of Technology) - teaching. These units enroll 80% of the 
undergraduate and 40% of the graduate students, generate over 2 1 ,000 full year equivalents 
(FYE) in instruction annually and include the efforts of over 1,000 faculty on the Twin Cities 
campus. Most of the resources in these colleges are devoted to personnel and supplies related to 
instruction. Any future growth or productivity gains would likely come from this area. 

One of the key facets in Incentives for Managed Growth was that colleges would be able to 
retain the tuition revenue generated by their course offerings and enrolled students. (The 
eventual university formula would be 75% of the tuition generated by courses and 25% 
generated by students.) This incentive magnifies the decisions made by the colleges and 
departments in Arts, Sciences, and Engineering, given their mission and financial structure. 
Clearly, collegiate deans and departmental administrators would be making daily planning and 
operating decisions that would directly affect their bottom line. Unfortunately, many faculty and 
administrators misunderstood the concepts of this new budgeting approach and assumed that any 
action that generated more instruction (regardless of cost or the impact on other departments) 
would lead to a positive outcome. The goal therefore was not just to generate and disseminate 
operational measures of instructional activity but to place these measures in a context that begins 
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to inform departmental and collegiate staff of the multiple dimensions of instructional resource 
management. Figure 1 was developed to demonstrate a number of related and relevant 
dimensions within Instructional Resource Management: 




Figure 1. A Decision Support Schema for Instructional Resource Management 



Three general dimensions (Demand, Productivity and Fiscality) are suggested with several 
examples of broad performance indicators arrayed by their respective spatial placement. This 
diagram has been useful in leading discussions with colleges and departments - they are familiar 
with many of the specific measures that are derived from these broad categories (e.g. course 
releases) and have begun to appreciate the cost and impact of their decisions. The creation of 
this general schema and the corresponding specific measures reflects in part the years of hands- 
on experience supporting operational planning and resource allocations as well as many 
discussions with collegiate and departmental staff at all levels and functions. Within each of 
these broad performance areas, specific indicators can be developed. Examples of several 
measures are included in Table 1. 
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Table 1. Examples of Specific Performance Indicators for Instructional Resource Management 



General Category 


Specific Performance Indicators 


Tuition Revenue Management 


Cross-College Student and Course Activity 


Instructional Cost Management 


Marginal Cost/Revenue by Course and Instructor 


Curricular Management 


Course Control Size vs. Average Section Size 


Enrollment and Tuition Management 


Net Tuition Yield by Admission Score 


Faculty Course Management 


Courses and SCH by Instructor FTE/Demo graphics 


Faculty Course Management 


Tuition Revenue Cost by Course Release 



This diagram is not meant to be exhaustive or definitive, but rather to emphasize the value added 
approach of grouping a number of familiar and relevant performance indicators in a schema that 
promotes understanding and awareness of the interaction of collegiate and departmental 
decision-making. 

CLARITY - Strategic System Solutions 

The focus of this paper has been on the need for collegiate and departmental operational 
performance measures and the necessity to disseminate this information in a manner that 
supports decision support. A detailed explanation of the technical solution, including front and 
back-end architecture and application development strategies can be found elsewhere (Boemer, 
Grotevant and Moloney, 1996; Grotevant and Moloney, 1996; Grotevant, Gentry and Moloney, 
1997). Nevertheless, for readers to fully appreciate the power of an information-rich 
environment at the desktop responsive to the needs of decision-makers, they are invited to 
explore the content and functionality that has currently been developed at the CLARITY web site 
http://www.clarity.umn.edu . Decision support is much more than data reporting - more than 
converting a paper report to an electronic report. Decision support is the creation of intuitive 
reports that reflect the daily operational questions that face collegiate deans and department 
chairs, that frame the issues in such a way as to promote discussion and understanding. Decision 
support includes giving users information at their desktop, integrated from multiple 
administrative databases, reconcilable to university level transactional reports, but including the 
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business rules and logic to reflect the reality of operations. Decisions support allows user 
functionality to aggregate and disaggregate data, select and manipulate variables, and explore 
exceptions all in a responsive and incisive manner. 

Summary 

Institutional researchers play a key role in the future success of higher education. Performance 
indicators and benchmarks are invaluable in communicating to external constituencies the 
positive and negative outcomes reflecting the administrative decisions. Aggregate, high-level 
benchmarks, however, are only part of the required performance measure landscape - decision 
, support for colleges and departments are also needed. This niche can be met but it requires a 
new breed of professional staff equipped with both a broad interdisciplinary experience in 
institutional research, budgeting, management, human resources and information. Combined 
with a wealth of knowledge in the daily operational decisions occurring in colleges and 
departments, these individuals are receptive to collegiate and departmental needs, and are trusted 
by these units. These professionals do not “fit” in central administrative units but their success 
in developing management tools can lead to the overall success in attaining institutional missions 
and goals. 
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